Innovation in assessment

Assessment for learning: guide for law teachers

This is part of a guide, compiled by Alison Bone (University of Brighton) and Karen Hinett (UKCLE) in 2002, providing an overview of the chief issues involved in assessment and how it affects learning and teaching in law.

---

Research into student learning has taught us much about the impact of assessment. We know that students learn best when assessment is:

• evenly timed
• represents or simulates real life
• the fear of failure is minimised
• perceived by students as relevant and appropriate to their needs as learners
  (Ramsden 1992, Entwistle et al 1987)

We know that individual learners tend to favour one learning style above another, and that some forms of assessment favour particular approaches. We also know intuitively as teachers that students take more seriously those topics which are to be assessed, and that learning and teaching are greatly enhanced when the assessment is seen to be integrated and relevant to the course, rather than tagged onto the end of a module as an afterthought.

Here we offer a few examples of more innovative forms of assessment, which can be integrated into courses and tailored to support the learning of content, skills, dispositions and appreciation of values as required.

The advantages of using a range of assessment techniques is that it gives students a chance to work to their strengths, whether this be team work or oral presentations. Variety in assessment encourages students to broaden their approach to learning and to develop different strategies.

Self assessment

Self assessment fosters the concept of development and reflection by elevating the status of the student to one of assessor and judge of quality in the same way as the lecturer. It places the student in a position where s/he must acknowledge strengths and weaknesses and be able to make plans for future improvement. It is based on the theory of experiential learning advocated by Lewin and developed by Kolb (1984) and Schön (1984), and allows the student to develop “the capacity to learn, the capacity to know how to learn, [and] the capacity to know what he has learned” (Heron in Boud, 1988: 78).

For Boud self assessment involves two key elements:

• the development of knowledge and an appreciation of the standards and criteria for meeting those standards
• the capacity to make judgements about whether or not the work meets those standards

Essentially, self assessment is about understanding what is required of a given piece of work and having the experience to make judgements about the quality of that work with reference to the criteria specified. It is concerned with the improvement and development of learning.

Sadler argues also that it is important that the student is able to monitor and make changes to work during the ‘act of production’ (Sadler, 1989: 121). The underpinning rationale of self assessment is that it helps the learner to understand what s/he is doing. It is a way of providing feedback which can be directly and immediately applied to the work in progress.

Learning cycles such as those developed by Schön (1987) describe a process of development which involves taking stock of existing knowledge, identifying ‘gaps’ and acquiring new knowledge which can be applied to future behaviour. Gaps in learning are traditionally filled by the tutor in the form of feedback.

Self assessment requires the student to identify the gaps and to give feedback on their work. However, making judgements about work involves having an understanding of the criteria and a concept of quality, and it also involves practice. The student must therefore have the knowledge and the ability to respond to feedback and make changes to work.

Self assessment for Sadler is a ‘downloading’ of evaluative experience from the tutor to the learner. This progressive enabling and passing over of responsibility from assessor to assessee may be seen as an ‘apprenticeship in judgement’.

Bridging the gap between traditional teacher led assessment and self assessment is ‘collaborative’ or ‘negotiated’ assessment’. This is an intermediate stage between dependency and autonomy of judgement; “a teaching and learning process in which the student and instructor meet to clarify objectives and standards” (Best et al in Somervell: 1993).

This process enables students to understand and communicate with the lecturer and their peers ideas about what they consider important, what should be assessed, and what weighting should be given to each specific criterion. It therefore provides students with the opportunity to engage in a dialogue about learning.

However, in order for dialogue to take place students need to able to understand the concepts of quality held by those making the judgements (Sadler, 1989). Students need to appreciate how the law underpins society and what it means to be a lawyer. To this end they need to be exposed to situations and experiences that allow them to ‘feel’, not just understand the law.

If we want our students to learn the importance of ‘fellow feeling’ and the habits of reflection as part of the transition from the undergraduate to the professional, we must construct learning experiences in which we model deliberation and reflection in action. If we want our students to think beyond the ‘box’ of law we cannot continue to treat them as empty shells. If we wish to imbue notions of justice and fairness in our students, which we argue is part of our work as law teachers, we need to make the vocabulary of justice, liberty, truth, and law part of our daily language as professionals.

— (LeBrun and Johnstone, 1994: 44)

This brings to our attention the need to provide opportunities to practice making judgements and to feel the responsibility of making a judgement against criteria. It means students are exposed to the subjectivity involved in making distinctions between quality work and, in legal terms, right and wrong.

Boud offers two arguments for involving students in self assessment and generating marks. The reality argument is based upon the view that learning is context and situation bound and therefore students need practice in interpreting requirements:

Students need to be able to assess themselves in situations in which they have only partial knowledge of the criteria to be used by others and when they may not fully accept the criteria which others will apply to them.

— (Boud 1989: 22)

The expediency argument is a result of the very real constraints felt by academics in meeting the heavy bureaucractic demands that many formal assessment systems generate. Boud argues that many staff do not have time to offer qualitative feedback to students and that self assessment is a way of reducing time spent on meaningless marking exercises.

If students mark their own work, either with respect to specified standards or their self established criteria, they not only release staff for more educationally worthwhile activities, but they are encouraged to reflect on their own work and the standards which can be applied to it.

— (Boud, 1995: 169)

Both arguments are based on educational principles and are concerned with improving the student’s understanding of the assessment process and how it can contribute to learning. However, a sceptic may consider both arguments to be quick and easy responses to modularisation, large student numbers and a diverse student body.

Self assessment may indeed help to alleviate these pressures but it is not a panacea for the ills of higher education. Introducing self assessment into programmes takes time, commitment and resources, and the benefits of such an approach are not always immediately recognisable. However, the evidence in law teaching is that where self assessment has been introduced with appropriate support, it appears to produce ‘deep’ learning and encourage students to strive towards achieving higher order skills.

Self assessment is not without problems. Issues of concern for academics and external stakeholders include accountability, quality and standards. Providing evidence of validity (that what is being taught is actually what is being assessed) is relatively straightforward, but offering evidence of reliability in a system where learners not only define their own criteria but assess their own work by that criteria is problematic.

The research into self assessment aims to dispel some of the concerns about reliability and reveals that rather than being simply an assessment tool, self assessment can be a learning experience in itself.

Self and peer grading

Existing literature on self assessment does not make the levels of responsibility afforded the learner clear. The methodology of existing research into self assessment varies. Some lecturers prescribe the criteria (Fry, 1990), others allow the students total freedom in their choice of criteria (Stefani, 1994) and a third group chose to collaborate with the students in the generation of assessment criteria (Falchikov, 1986). It would appear from the results of the research that the extent to which the student is engaged in the process of generating assessment criteria directly affects the likelihood of agreement between tutor and student of grades.

Boud and Falchikov conducted a major review of the literature on self assessment in 1989. The review involved analysing 48 quantitative studies concerned with the differences between student and staff ratings of performance. They investigated the hypothesis that under-achievers tend to over-rate themselves whilst high achievers tend to under-rate themselves, but found no overall consistent tendency to over or underestimate performance.

They also noted that there was considerable methodological variation in the research, and suggested that no conclusions could be drawn concerning the improvement in self assessment ability over time or with practice, as there were too few studies investigating this variable.

Stanton (in Falchikov and Boud 1989) found that when staff and students selected the assessment criteria through a process of mutual collaboration there was an 80% agreement of grades. This would suggest that whilst other factors such as experience, maturity and simplicity of task may play an important part in securing agreement of grades, perhaps the single most important factor is the role the students play in the selection of criteria.

Boud and Falchikov support this view, suggesting that the degree to which criteria are made explicit influences the correspondence between student and teacher marks. Familiarity with, and understanding of, these criteria also enhances student-teacher agreement.

It would appear that it is not enough simply to make the criteria explicit, but that students have to have some sense of ownership of the criteria and to fully understand why they are in place.

Implicit in the benchmark statements and the QAA framework is the need to make assessment criteria available to students. Availability, however, does not equal understanding. It is possible that institutions may favour availability over transparency, which will do little to improve student learning.

A crucial factor of self assessment is that it allows students the opportunity to question and discuss the importance of criteria. Research has identified an incongruence between the criteria valued by staff with those of students. Traditional assessment is based on an end product, however research at Northumbria University reveals that students wish to be rewarded for the process of learning. In particular they consider the reward of effort to be a central part of the assessment process.

Becker et al recorded this dichotomy between performance and effort as early as 1968 and claimed that while lecturers tended to see student success as deriving from ability plus motivation, students tended to view academic success as largely dependent on time and effort. (Becker et al in McDowell, 1995: 307)

From their literature review Falchikov and Boud (1989b, p425) highlight the quality of design of the study, level of the course of which the assessment is a part, and the area of study itself as variables likely to produce consensus between staff and students. Interestingly, they propose that the nature of the task is not as important as these other factors, which would suggest that self assessment lends itself to particular disciplines rather better than to others. What studies do suggest is that self assessment is successful because it is perceived by the students as ‘real’.

Self assessment is more successful when the students perceive the assessment to be interlinked with their learning and when they can make connections between the assessment and the discipline. It is therefore likely that students on programmes of professional development or training such as law, where learning objectives are tightly defined by professional bodies, find making connections between experience and self assessment easier than students in more traditionally ‘academic’ subjects such as philosophy.

A criticism of self assessment studies is that whilst well intended they tend to look at ways of increasing the effectiveness of learning and not to measure participants’ attitudes towards self assessment or to examine the effects of self assessment on learning (Falchikov and Boud, 1989b, p426). It may be more useful to develop ways in which systematic formative self assessment activities can be incorporated into courses to improve student skills in making sensitive and aware judgements on their own work (Falchikov and Boud, 1989: 532).

Research into the validity of self-grading is important, especially for purposes of validation and evidence of standards, however it is perhaps more fruitful from a pedagogical point of view to recognise the benefits of self assessment in providing an apprenticeship in judgement and decision making (Boud and Lublin, 1983).

Reflective journals

Atkins argues that one of the key characteristics of a critical learner is “to be able to reflect on one’s own practice and to use feedback to assess and manage one’s own performance” (1995). Self and peer assessment provide a framework for this development to take place.

Providing guided but direct and authentic evaluative experience for students enables them to develop their evaluative knowledge, thereby bringing them within the guild of people who are able to determine quality using multiple criteria. It also enables transfer of some of the responsibility for making decisions from teacher to learner. In this way, students are gradually exposed to the full set of criteria and the rules for using them and so build up a body of evaluative knowledge. It also makes them aware of the difficulties which even teachers face of making such assessments, they become insiders rather than consumers.

— (Sadler 1989:135)

We may agree with this in principle but how then do we begin to assess the quality of reflection? One easy suggestion is that we do it in the same way that we assess all other skills; by reference to clear criteria, providing feedback about areas of development. However this does not entirely provide for the complex nature of reflection. Students who have written about their experiences or difficulties with an academic piece of work are likely to have exposed personal beliefs and values as well as the extent of their knowledge. As McGill and Brockbank (1998) explain the fact that reflection draws on both the cognitive (knowledge and skills) and meta-cognitive (behaviour and feelings) is a great strength: an assessment system which addresses reflective learning should do so in terms of both outcome and process.

Traditional assessment has concentrated on providing evidence of the product and outcome of learning. However, providing evidence of process is more challenging. McGill and Brockbank suggest that evidence is collected in terms of self report and other report.

Other reports might include learning contracts and evidence from a mentor or practice tutor (in work based learning). This evidence can be used to display development and the ability to cope with increasingly difficult learning situations in different contexts. Self report might include reflective diaries and logs and reflective evidence which can be collated in a portfolio of evidence. Learners are encouraged to reflect on the quality of learning and to indicate ways in which they might improve. These self reports can either be private or public documents which means students can differentiate between what is ‘ego involving’ and ‘task involving’ (Black and Wiliam, 1998). McGill and Brockbank (1998) suggest that when ‘self report’ and ‘other reports’ are confirmed by the judgement of the tutor some claim to triangulation and validity can be made. What this means is that students are co-dependent on tutors, peers and their own self-knowledge to provide the feedback they need for improved learning.

Oral presentations

Oral work is not given much currency in the literature on legal education. Examples of advocacy and interviewing skills are more common in professional legal education but there is a dearth of accessible examples of assessed presentations or assessed discussion work at undergraduate level (let us know if you use it!) Discussions, presentations and student-led seminars are all taking place in UK law schools, but there is a lack of written material describing how such activities are assessed. This is likely to change with the introduction of benchmarking. The documentation clearly values oral communication as a key skill:

Law students are expected to be good at both written and oral communication. Whereas written communication is assessed heavily by formal examinations, oral communication is demonstrated by a variety of compulsory and voluntary activities, for example tutorial performance or mooting.

— (QAA, 2000:7)

Most of us agree that oral communication is a key skill for lawyers; what is not made clear is how oral communication should be assessed. Again we return to issues to function and purpose of assessment. Are we assessing content knowledge or the ability to articulate an argument in front of an audience? Research into student learning suggests that there are a number of pedagogic reasons for using oral communication as a way of enhancing learning (Sambell et al 1996, Hinett 1997, Joughin 1999).

Students from a range of disciplines were asked; “how do you know when you have learnt or understood an aspect of the programme you study?”. Anticipated responses included “when I get a good grade”, however the most common answer was, “when I can talk about it.” It seems students evaluate their understanding by their ability to able to talk about the subject or explain it to someone else. As educators we are well aware of the steep learning curve that takes place when we try to structure information in a way which can be easily communicated. We should not be surprised to learn that students also use oral articulation as a benchmark for understanding. There appears to be a connection between speech and understanding. This view is illuminated by an exchange between ateacher and a student speaking about assessment. The teacher reminded the student that she had performed an oral presentation as part of her assessment and she replied:

Yes, but that didn’t feel like it was being assessed. Somehow it had a different feel to it. To me work signifies exams and writing things down.

Joughin hypothesises that oral assessment is more likely to lead to a deep approach to learning since it requires a personal commitment and engagement with subject matter. Oral assessment is not perceived as an easy option; in fact studies reveal that students find oral assessment and presentations difficult. Students allocate additional time to the preparation of a presentation and express feelings of apprehension about their ability to express themselves coherently. This is partly due to inexperience in oral assessment but has more to do with the perception of an ‘audience’. Education involves interaction between a number of ‘agents’ of learning. Agents might be tutors, placement teachers, professional practitioners or peers, all of whom influence the learning potential of an individual. Students perform in the knowledge that their work is being judged by these agents and if the form of assessment is oral and face to face then the pressure to perform well is greater. Ignoring the problems of self confidence, several factors emerge about students’ perception of oral assessment that may explain the adoption of a deep approach:

• students do not wish to make a fool of themselves in front of those passing judgement which prompts a responsibility towards work
• speech is transparent to waffle and padding
• in order to answer questions on a subject the student has to understand for him or herself (reducing the opportunity for plagiarism)
• speaking inevitably means that you are heard (eradicating any possibility of confused areas being overlooked by those making judgements)
• oral assessment involves body language conveying more about the level of comprehension than can be expressed in written form
• there is a personal involvement and ownership of the spoken word
  In written assessment it is possible for students to submit and deny responsibility for their work. The power differential between ‘learner’ and ‘assessor’ ironically enables the student to disengage from the learning process. One student remarked:

You only have to hand it in and you don’t have to be there when it’s actually marked. You’re separated from your piece of work because you’re not there.

The physical distance created between the student’s production of written work and the assessor’s judgement appears to promote an intellectual separation in the mind of the student. Improving student learning therefore necessitates a closure of the perceived gap between feedback and action.

References

Last Modified: 4 June 2010